Reshading Light Fields

A category of methods usually called Image Based rendering has appeared recently to produce realistic images of objects. Instead of creating models from geometric primitives which are then rendered according to the point of view and the illumination given, the light reflected/emitted from objects is captured from enough different points of view to be able to reconstruct the light received from the objects from any new point of view. The resulting data structure is usually called a light field. This approach has many advantages, but also many disadvantages, the most obvious one being that the data has been captured under a specific illumination condition, and there is very little information to be able to reshade the objects, and therefore to be able to insert them into arbitrary new environments. We propose here a series of techniques which allows reshading light fields. We assume that there is no depth information available with the light field, and our steps consists of the following: (1) determining a coarse geometric shape of the object by creating an octree of cells occupied by the object, (2) estimating surface normals from the boundary points so determined, (3) estimating the reflectance properties by making simplifying assumptions about the surface reflectance function. Once these steps are done, one can insert the object within standard geometric models, determine visibility and shade the whole scene (including shadows and intereflection if necessary) to produce images where light fields and geometric objects are integrated. We have only began the implementation work, and this paper is a research plan rather than a report of results. In it we give the motivation and backgroundfor our application, and describe the techniques we plan to use, and give the reasons we selected them.